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1  Documentation  Roadmap 

This  Final  Report  is  organized  into  the  following  sections: 

•  Section  1  (“Documentation  Roadmap”)  provides  information  about  this  document  and  its  intended 
audience.  It  provides  the  document  overview  and  describes  the  content  of  each  section. 

•  Section  2  (“Program  Overview/Review”)  provides  a  summary  of  schedules  and  milestones 
achieved  during  the  year  by  phase. 

•  Section  3  (“Base  Year  Technical  Overview/Review”)  a  technical  overview  of  each  phase,  from  an 
architectural  and  implementation  perspective.  Interaction  with  other  ONR  performers  and  testing 
accomplished  in  each  phase  are  also  summarized. 

•  Section  4  (“Results,  Conclusions  and  Next  Steps”)  summarizes  overall  status,  lessons  learned  and 
presents  goals  and  objectives  for  the  first  option  year. 

•  Section  5  (“Directory”)  provides  reference  information,  including  a  glossary  and  acronym  list. 

1.1  Document  Management  and  Configuration  Control 
Information 

•  Revision  Number:  - 

•  Revision  Release  Date:  08/14/09 

•  Purpose  of  Revision:  Initial  Release 

•  Scope  of  Revision:  N/A 

1.2  Purpose  and  Scope 

This  Final  Report  provides  a  summary  of  work  done  under  the  ONR  Transparent  Urban  Struc¬ 
tures  contract  in  the  Base  Year  of  the  Transparent  Urban  Structures  program  by  the  team  com¬ 
prised  of  Natural  Selection  Inc,  the  U.S.  Army  Engineering  Research  and  Development  Center 
(ERDC),  Textron  Defense  Systems  and  the  anthropologist  Liam  Murphy.  The  effort  was  divided 
into  3  phases,  and  the  programmatic  and  technical  discussions  will  address  each  phase  individu¬ 
ally. 
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2  Program  Overview/Review 
2.1  Schedules 

Figure  1  shows  the  executed  schedule  per  phase  for  the  Base  year. 
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Figure  2-1  Schedules  by  Phase 
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Table  2.1  Summarizes  statused  milestones  for  the  program  Base  year. 
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APR 

MAY 
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Phase  3  TIM  and  minutes 

AUG 

AUG 

AUG 
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AUG 

AUG 

AUG 
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Monthly 
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Final  Report 
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Table  2-1  Statused  Milestones 


2.2  Meetings  Summary 

The  base  year  program  had  a  defined  set  of  periodic  meetings  that  are  all  documented  with  meet¬ 
ing  minutes.  These  meetings  included: 

•  Weekly  Team  meetings  with  subcontractors 
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•  Weekly  program  meeting 

•  Monthly  program  review  with  Textron  Senior  management 

•  Monthly  Teleconference  with  ONR:  Minutes  have  been  distributed  to  ONR 

•  3  Technical  Interchange  Meetings  with  ONR:  Minutes  have  been  distributed  to  ONR 

•  Bi-weekly  anthropology  meetings  in  Phase  3  with  subject  matter  experts  (SMEs)  and  de¬ 
velopment  team 

•  Daily  integration  meetings  towards  the  end  of  each  phase 

2.3  Documents 

Monthly 

•  Monthly  Spend  Report  9/2008  -  7/2009 

•  Monthly  Financial  Report  9/2008  -  7/2009 

•  Monthly  Internal  Review  Slides  and  Meeting  Minutes  9/2008  -  7/2009 

Contractual 

•  Textron  /ONR  SOW  TS-W8C03  (AV04)  8/1 2/2008 

•  Contract  No.  N00014-08-C-0244  MOD-l-NONE-Initial  8/14/2008 

•  Contract  MOD  2-P00001  -Initial  -1/1 5/2009 

•  Contract  MOD  3-P00002 -Final-6/1 6/2009 

Technical  Interchange  Meetings 

•  TIM  I  -  Presentation  and  Meeting  Minutes  2/1 8/2009 

•  TIM  II  -  Presentation  and  Meeting  Minutes  6/5/2009 

•  RBIE- Trade  Study  6/10/2009 

•  Ramadi  Scenario  Playbook  3  6/10/2009 

•  TIM  III  -  ISR  Data  Analysis  Workshop  Slides  and  Meeting  Minutes  8/5/2009 

Deliverables 

•  ONR  TUS  Software  Architecture  Specifications,  TUS-SAD-0001,  10/13/2008 

•  Experiment  Test  Plan  -Phase  I,  TUS-ETP-0001 -Initial,  1/19/2009 

•  Experiment  Test  Plan-Phase  II  TUS-ETP-0001 -REV  B-  8/14/2009 

•  Experiment  Test  Plan  Phase  III  TUS-ETP-0001 1  -REV  C-8/1 4/2009 

•  ONR  TUS  Integrated  Program  Plan  TUS-IPP-01  Initial-  9/12/2008 

•  Textron  Base  Year  Final  Report,  TUS-FIN-0001,  8/14/2009 
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3  Base  Year  Technical  Overview/Review 

The  system  being  developed  by  the  Textron  team  on  the  Transparent  Urban  Structures  Program  is 
comprise  of  a  software  framework  infrastructure  that  integrates  three  key  services  and  a  simula¬ 
tion  environment  into  a  blended  Service  Oriented  Architecture,  Event  Driven  Architecture 
(SOA/EDA)  software  communications  architecture. 

The  base  year  effort  is  comprised  of  3  phases  of  development  with  increasing  functionality  and 
interoperability  of  the  system. 

3.1  Requirements 

The  following  are  the  high-level  technical  requirements: 

1 .  Interoperability  with  DoD  Frameworks  and  Architectures 

a  Develop  a  blend  of  EDA  and  SOA  architectural  approaches.  EDA  will  allow  for  near 
real  time  messaging  in  the  system  while  the  SOA  paradigm  enables  a  pull  back  feature 
that  will  allow  the  request  of  data  via  a  XML/SOAP  web  service  interfaces, 
b  Implement  a  common  messaging  bus  for  data  communications  amongst  services  in  the 
system 

c  Expose  datasets  via  extensible  markup  language,  simple  object  access  protocol 
(XML/SOAP)  web  services 

2.  Geo-Cultural  service  to  provide  context  and  actionability  to  sensor  data  and  intelligence 

d  Data  collection 

e  Possible  data  fusion  for  heterogeneous  data  sources 
f  Inferencing  engine  to  determine  if  data  is  actionable 

3 .  Integrate  with  multiple  sensor  systems  and  technologies 

g  Develop  M&S  environment 

i  Urban 

ii  Multiple  sensor  modalities 

4.  Scalable  to  option  years 

3.2  Requirements  Coverage 

These  requirements  have  been  addressed  in  the  following  fashion: 

1 .  Web  based  semantic  web  technologies  were  used  to  facilitate  data  query  and  dissemination.  Protege, 
SPARQL,  Sesame  and  JAVA  are  technologies  implemented  on  this  program.  These  technologies  pro¬ 
vide  SOA  tools  to  encourage  interoperability.  Common  messaging  bus  (ActiveMQ)  was  implemented 
to  aid  data  flow  between  components  of  the  systems.  This  bus  satisfied  the  EDA  requirement  of  the 
system. 

2.  Geo-cultural  components  were  implemented.  This  includes  a  geo-database,  ontology,  instance  data 
and  an  inferencing  engine.  Additionally  the  ontology  is  being  expanded  to  encompass  anthropological 
relationships.  The  inferencing  engine  (Drools)  chosen  for  our  system  is  able  to  produce  output  on  ac- 
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tionable  data.  Future  iterations  of  the  system  may  have  more  robust  Bayesian  fusion  rules  integrated 
into  the  platform. 

3.  Integrated  simulation  system  in  phase  1  that  mimicked  urban  sensors  in  a  tailored  scenario.  Showed 
the  ability  to  use  sensor  data  in  real  time,  which  could  be  fused  with  the  ontological  and  geographic 
data.  As  work  continues  with  other  ONR  performers,  leveraging  of  more  live  data  is  expected. 

4.  Scalable  to  option  years  implies  two  philosophies.  First,  that  the  architecture  for  the  system  builds  on 
itself  over  time  in  a  fashion  that  can  support  a  wide  range  of  inputs  and  interfaces  to  other  systems  in 
an  environment  of  increasing  complexity  and  functionality.  Second,  that  the  Textron  system  comply 
with  the  architectural  guidelines  and  implementation  standards  posed  by  ONR,  while  providing  in¬ 
formation  services  to  other  performers  and  ultimately  the  warfighter.  Both  of  these  have  been  firm 
tenants  of  this  design  and  priority  has  been  given  to  working  with  other  performers  and  understanding 
the  objective  ONR  architecture. 

3.3  Phase  1 

Figure  3-1  shows  the  system  architecture.  The  colored  components  will  be  developed  under  the 
Base  Year  of  the  current  contract.  The  grayed-out  components  are  Option  Year  development  ef¬ 
forts.  As  shown  with  dotted  lines,  some  components  developed  in  the  Option  Years  will  replace 
components  developed  in  the  Base  Year.  The  colored  components  in  the  red  rectangle  were  de¬ 
veloped  and  tested  in  Phase  1 .  The  following  section  describes  these  components. 


Figure  3-1  Phase  1  Architecture 


ActiveMQ  Message  Bus  -  Provides  the  coupling  of  all  the  disparate  services  through  topic-based 
publish  and  subscribe  capabilities. 


Use  or  disclosure  of  the  data  contained  on  this  sheet  is  subject  to  the  restriction  on  the  title  page. 

6 


TEXTRON  Defense  Systems^ 

TEXTRON  Systems 

Simulation  -  A  High  Level  Architecture  (HLA)  based  environment  that  models  &  simulates  an 
urban  geographic  area,  including  building,  roads,  foliage,  people,  vehicles,  sensor  systems,  and 
more.  The  sensors  in  this  environment  publish  data  to  the  ActiveMQ  Message  Bus  via  an 
adapter.  In  the  future,  the  simulation  will  be  able  to  subscribe  to  topic  data  on  the  ActiveMQ 
Message  Bus. 

Sensor  Information  Collection  (SIC)  Service  -  Subscribes  to  formatted  sensor  data  topics  on  the 
ActiveMQ  Message  Bus  and  stores  the  data  for  other  uses.  There  is  a  flat  configuration  file  that 
contains  the  database  schema.  In  the  future,  the  SIC  will  publish  data  to  the  ActiveMQ  Message 
Bus  as  well  as  through  an  SOAP/XML  web  service  interface. 

XML  Translator  Service  -  Subscribes  to,  and  translates,  raw  sensor  data  into  standard  XML  for¬ 
mats  and  then  publishes  formatted  data  to  the  ActiveMQ  Message  Bus 

3.4  Phase  2 
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Figure  3-2  shows  the  Phase  2  system  architecture.  The  colored  components  will  be  developed 
under  the  base  year  of  the  current  contract.  The  grayed-out  components  are  option  year  devel¬ 
opment  efforts.  As  shown  with  dotted  lines,  some  components  developed  in  the  option  years  will 
replace  components  developed  in  the  base  year.  The  colored  components  in  the  red  rectangle 
were  developed  and  tested  in  Phase  1 .  The  colored  components  in  the  blue  rectangle  were  de¬ 
veloped  and  tested  in  Phase  2  and  are  described  below. 


Geo-Cultural  Service  -  this  service  has  three  building  blocks: 

Geo-Cultural  Ontology  -  The  geo-cultural  ontology  was  created  prior  to  this  project  and 
is  undergoing  constant  improvement  or  enhancement.  It  is  a  Web  Ontology  Language 
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(OWL)  file  containing  relationships  between  geographic  entities  and  cultural  practices, 
including  activities  and  events. 

Geo-Cultural  Knowledge  Store  —  The  knowledge  store  contains  instance  data  (dates, 
times,  etc.)  for  when  particular  activities  and  events  occur  in  the  geo-cultural  area  of  in¬ 
terest.  While  the  OWL  file  could  represent  instances  of  the  data,  the  knowledge  store  de¬ 
couples  the  instance  data  from  the  ontology  to  improve  scalability  of  the  system. 

Geo-Database  -  The  geo-database  for  Ramadi,  Iraq  was  provided  by  the  Navy  Research 
Lab  (NRL).  It  is  used  in  conjunction  with  the  Ontology  and  Knowledge  Store  to  provide 
the  rule-based  inferencing  engine  with  contextual  data. 

Protege  Utility  to  convert  ONR  ontologies  into  Java  classes  —  The  existing  ONR  ontologies  rep¬ 
resent  a  common  framework  for  performers  to  interpret  the  interaction  of  data.  It  is  necessary  to 
leverage  these  ontologies,  so  as  to  contain  the  same  information  when  interacting  with  other  per¬ 
formers.  Protege  contains  a  utility  that  allows  a  user  to  export  an  OWL  file  to  Java  classes.  These 
classes  follow  a  format  such  that  the  name  of  the  ontology  (e.g.,  “Person”)  will  be  an  interface, 
and  the  implemented  ontology  containing  the  properties  specific  to  that  ontology  will  be  con¬ 
tained  within  the  “impl”  package  of  the  high-level  ontology  package,  with  the  same  name  except 
preceded  by  “Default”  (e.g.,  “DefaultPerson”). 

Scenario  Configuration  Environment  -  This  file  contains  all  events  associated  with  a  given  sce¬ 
nario,  including  events  that  generate  potential  targets  and  threats  for  the  scenario.  This  file  also 
contains  periphery  information  such  as  date,  time,  weather,  and  interfaces  with  the  Sys¬ 
tem/Graphical  User  Interface. 

System/Graphical  User  Interface  -  The  User  Interface  performs  the  essential  function  of  present¬ 
ing  data  to  the  user  in  a  visually  condensed  but  intuitive  fashion.  This  is  achieved  by  performing 
as  many  actions  on  the  backend  as  possible,  such  as  loading  all  scenarios  and  rules  immediately 
on  startup,  as  well  as  displaying  each  asserted  fact  and  fired  rule  (and  potential  threats  and  tar¬ 
gets)  in  the  appropriate  text  box  at  the  appropriate  time  (immediately  following  the  user’s  asser¬ 
tion  of  the  next  fact). 

Scenario  Configuration  Input  Utility  -  This  utility  is  responsible  for  correctly  configuring  the 
initial  setup  conditions  of  the  system  by  reading  in  all  buildings  and  scenarios  contained  within 
the  scenario  playbook. 

Rule-Based  Inferencing  Engine  (RBIE)  -  The  RBIE  contains  rules  derived  from  military  and  an¬ 
thropological  subject  matter  experts.  The  inferences  made  based  on  these  rules  help  create  lists 
of  targets  and  threats  for  a  given  scenario,  and  will  provide  alerts  when  a  threshold  threat  level 
(or  similar)  is  reached. 

Resource  Description  Framework  (RDF)  Statement  Generator  -  This  utility  is  responsible  for 
correctly  assembling  and  publishing  RDF  statements  to  a  specified  server. 
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Figure  3.3  illustrates  the  usage  of  the  Phase  2  system.  The  accompanying  text  will  summarize 
the  functionality  of  each  box  in  the  diagram  in  terms  of  inputs  and  user  interface. 


Figure  3-3  Phase  2  System  Usage 


System  Inputs 

The  input  components  of  the  Phase  2  System,  illustrated  above,  are  solely  responsible  for  provid¬ 
ing  context  to  the  system.  These  components  drive  the  system  functionality  and  their  interactions 
are  displayed  on  the  Graphical  User  Interface  (GUI) : 

Scenario  Playbook 

The  scenario  playbook,  shown  in  figure  3-1,  contains  contextual  events,  such  as  socio-economic 
status,  military  support,  or  non-insurgent  illegal  activity,  and  intelligence  events,  such  as  arrests 
and  recorded  sensor  events  (i.e.  Metallic  Anomaly).  This  provides  a  way  to  script  scenario  his¬ 
tory,  events  and  detections  for  testing  the  system. 
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Figure  3-4  Scenario  Playbook 
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The  Contextual  Events  are  past  events  that  serve  to  give  context  to  the  current  Scenario,  and  In¬ 
telligence  Events  are  meant  to  simulate  events  that  occur  in  real-time.  Each  Intelligence  Event 
contains  the  following  fields  (table  3-1): 


Field 

Description 

Type 

Intel  or  Sensor  Data  event 

What 

Event  sub-types,  such  as  Pending 
Attack  or  Metallic  Anomaly 

Building  Type/Location 

Building  Type/Location 

Building  ID 

Building  ID 

Location  Descriptor 

Exact  or  Non-Exact 

Addt'l  Info 

Additional  Comments  pertaining 
to  Intelligence  Event 

Classification 

Further  Classification  of  the  In¬ 
telligence  Event 

Date 

Date 

Time 

Time 

State  Change 

This  field  reflects  the  what  the 
output  of  the  Intelligence  Event 
should  be,  and  is  not  processed  as 
input. 

Table  3-1  Event  Fields 
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Instance  Database 

The  instance  database  contains  local  geo-cultural  information  for  the  specified  region:  in  this 
case,  Ramadi,  Iraq.  This  data  and  its  inter-relationships  are  described  by  the  geo-cultural  ontol¬ 
ogy.  Instance  data  can  be  queried  by  the  inferencing  engine  as  part  of  its  execution  thread  or  ul¬ 
timately  by  outside  users  and  services. 

Geo-database 

The  geo-database  contains  information  regarding  locations  with  the  ability  to  represent  objects,  build¬ 
ings,  roads  and  attributes  associated  with  them.  Each  location  contains  a  unique  index  (the  Scenario 
Playbook  assumes  that  the  location  has  been  identified  via  some  external  means),  with  a  location 
or  building  functionality,  such  as  ‘DefenseActivities’  or  ‘MunitionsSupplyFactory’.  This  par¬ 
ticular  geo-database  is  derived  from  one  received  from  NRL  for  Ramadi  Iraq.  It  has  been  en¬ 
hanced  for  the  South  West  portions  of  Ramadi  in  support  of  this  particular  scenario. 

Rules  File 

The  Rules  file  contains  inferencing  ‘rules’  that  contain  a  subject  and  a  predicate.  Each  rule  repre¬ 
sents  a  cause-effect  relationship  between  the  inputs  to  the  system,  contextual  and  intelligence 
events,  and  the  outputs  of  the  system,  threat  and  target  levels.  In  this  light,  the  subject  is  the  input 
to  the  system,  such  as  a  metallic  anomaly  occurring  at  a  recorded  location,  and  via  the  associa¬ 
tions  drawn  from  the  instance  database  and  geo-database,  the  predicate,  or  output  of  the  event,  is 
that  the  threat  level  for  the  given  location  is  increased. 

For  the  Phase  2  System,  the  RBIE  selected  was  a  JAVA  based  solution  named  Jess,  based  on  it’s 
interoperability  with  Protege.  Issues  occurred  in  Phase  2  with  Jess’s  ability  to  handle  complex 
data  types.  This  issue  is  addressed  in  Phase  3. 

User  Interaction  with  the  GUI 

These  components  provide  the  user  with  the  means  of  asserting  facts  into  the  Inferencing  Engine, 
and  provide  back  to  the  user  the  result  of  a  subset  of  rules  firing.  See  figure  3-5. 

User  Presses  Button  to  Assert  Fact 

The  following  figure  represents  the  Jess  GUI: 
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Figure  3-5  Jess  GUI 


The  user  must  first  press  the  “Start”  button  to  start  the  Rete  algorithm  based  Jess  engine,  and 
then  can  press  the  “Assert  Fact”  button  as  desired  to  observe  assertion  of  facts  and  the  firing  of 
rules. 

Fact  is  Asserted 

When  the  user  pressed  the  “Assert  Fact”  button,  the  next  Intelligence  Event  in  the  ‘Intelli- 
genceEventVector’  is  asserted  into  the  Jess  RETE  engine.  This  use  case  allows  the  user  to  see 
the  output  of  each  rule  (or  rules)  firing,  and  how  that  impacts  the  overall  system. 

Rules  are  Fired 

For  a  given  fact,  a  subset  of  the  total  number  of  rules  will  fire,  where  that  subset  is  between  0  and 
the  total  number  of  rules.  Thus,  though  unlikely,  an  asserted  fact  could  cause  every  single  rule  to 
fire,  or  could  cause  none.  When  a  fact  is  asserted  that  meets  the  criteria  of  the  subject  of  a  rule, 
then  the  predicate  of  the  rule  is  executed.  For  instance,  if  an  ‘Arrest’  event  occurs  (subject),  then 
the  Threat  Level  of  that  person’s  organization,  based  on  their  Socio-Economic  Status  and  previ¬ 
ous  attack  history  will  be  elevated  (predicate).  The  output  of  a  subset  of  rules  firing  is  updating 
the  GUI  to  correctly  represent  the  underlying  information. 

GUI  is  Updated  appropriately 
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The  process  between  the  assertion  of  facts  and  firing  of  rules  is  transparent  to  the  user.  The  user 
simply  clicks  “Assert  Fact”  and  the  system  responds  accordingly.  See  figure  3-6: 
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Figure  3-6  Asserted  Fact  in  Jess  GUI 


Though  the  “Top  Threats,”  “Top  Target  Risks,”  and  “Rules  Fired”  panels  have  not  updated  in  the  above 
view,  they  would  serve  to  represent  the  top  Target  and  Threat  Levels  that  have  been  deduced  by  the  rule- 
based  inferencing  engine  and  the  report  of  which  rules  have  been  fired  would  appear  in  the  “Rules  Fired” 
panel. 


3.5  Phase  3 

Efforts  in  Phase  3  have  focused  on  four  key  areas: 

•  Selection  and  Integration  of  a  new  RBIE 

•  Developing  functionality  to  interoperate  within  the  objective  ONR  Architecture  including  support 
to  the  August  2009  Data  Analysis  Demo  (DAD)  integration  and  test  effort. 

•  Support  to  the  September  2009  ISR-C2  demonstration 

•  Initial  development  of  an  Anthropological  Ontology 
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3.5.1  Selection  and  Integration  of  a  new  RBIE 

Phase  3  presented  the  need  to  revaluate  the  inferencing  engine.  The  Jess  engine  that  was  used  in  Phase  2 
did  not  provide  the  ability  to  access  nested  data  types  in  a  manner  that  was  easy  to  the  developer.  Seeing 
as  most  of  the  structures  generated  from  the  ONR  ontology  are  hierarchal  in  nature  it  was  determined  that 
a  new  direction  was  needed  for  the  RBIE.  Additionally  Jess  provided  zero  debugging  capabilities.  Nu¬ 
merous  RBIEs  were  evaluated  in  the  process.  Table  4  lists  these  RBIEs: 
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Platform 

Comments 

Manadrax  Inferencing  Engine 

No  Protege  plug-in 

No  debugging  capability 

Last  update  2008 

Algernon 

Protege  plug-in 

No  debugging  capability 

Last  update  2005 

JTP 

No  Protege  plug-in 

Last  update  2004 

JBOSS  Rules  (Drools) 

JAVA  based 

Protege  plug-in 

Good  debugging  capabilities 

Widely  supported 

Last  update  May  2009 

JENA 

Protege  plug-in 

No  debugging  capabilities 

Last  update  2009 

Zilonis 

Not  Widely  supported 

Table  3-2  RBIE  Tradespace 


The  decision  was  ultimately  made  to  go  with  Drools  due  to  its  eclipse  plug-in,  support  for  all  Java  types, 
good  debugging  capabilities  Java  based  rules,  large  community  base,  support  of  Rete  chaining  algorithm, 
and  its  suite  of  visualization  tools.  Drools  is  currently  integrated  into  and  tested  in  the  Textron  environ¬ 
ment. 
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3.5.2  Objective  ONR  architecture  and  the  DAD  demonstration 

Figure  3.7  illustrates  the  Objective  architecture  picturing  the  Textron  team’s  services  participating. 


Sensor  Data 
-COMINT 
-DNE 
-Geo  IWT 
-Facillty  INT 
HUMINT, 
Census 
•Open  Source 
-IMINT,  WAS 
-Biometrics 
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-Cultural 
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Applicaiion 

Service 
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fl 

ri 
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Sensor  Planning  and  Management 


Figure  3-7  ONR  Objective  Architecture 

The  Geo-cultural  Fusion  and  Inferencing  Service  interacts  within  the  architecture  as  follows: 

•  Outputting  RDF  to  the  RDF  repository 

•  Inputting  user  queries 

•  Receiving  data  from  other  services 

•  Outputting  data  to  other  services 

•  Interacting  with  other  services 

•  Receiving  sensor  metadata 

•  Receiving  raw  sensor  data 

•  Outputting  Sensor  Planning  RDF 

Efforts  with  regards  to  the  DAD  demo  are  a  first  step  towards  realization  of  this  architecture.  Figure  3.8 
shows  the  Phase  3  usage  and  illustrates  the  implementation  of  the  functionality  to  support  interaction  with 
the  RDF  repository. 
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Figure  3-8  Phase  3  Usage 


System  Input  Descriptions 

The  input  components  of  the  Phase  3  System  provide  scenario  sequencing,  geographic  and  geo- 
cultural  data  and  the  inferencing  rules.  Each  is  described  below: 

Scenario  Playbook 

While  Phase  3  testing  has  been  done  with  the  same  playbook  as  Phase  2,  development  of  a  new 
playbook  has  begun  to  support  the  DAD  demonstration.  Based  on  the  classified  NRL  data  pro¬ 
vided  by  NRL,  it  will  provide  a  relevant  Afghanistan  scenario  for  current  demonstration  and  fu¬ 
ture  development  efforts. 

Instance  Database 

As  scenario  data  was  delayed  in  delivery,  an  instance  database  has  been  under  development  for 
the  DAD  demonstration  using  unclassified  sources. 

Geo-database 

While  NRL  provided  the  beginnings  of  a  geo-database  for  the  DAD  demonstration,  additional 
data  is  needed  for  complete  support  of  the  demonstration.  Some  of  this  will  have  to  be  manually 
generated  based  on  the  requirements  provided  by  the  actual  content  of  the  data. 
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Rules  File 

The  functionality  of  the  rules  file  has  not  changed  between  Phase  2  and  Phase  3.  Additional 
rules  are  continuously  being  developed.  However,  the  Phase  3  system  utilizes  Drools  rather 
than  Jess  as  its  inferencing  engine. 


System  Console  Display  Events 

Fact  is  Asserted 

The  Phase  3  System  does  not  make  use  of  the  Jess  GUI,  and  as  a  result,  there  is  no  interface  that 
the  user  may  utilize.  Thus,  the  assertion  of  facts  occurs  once  the  scenarios  have  been  loaded  into 
the  system.  The  user  may  watch  what  facts  are  asserted  by  watching  the  system  console  as  they 
occur,  but  has  no  control  over  when  they  assert  (as  with  Phase  2  System). 

Rule  is  Fired 

As  with  the  assertion  of  facts,  rules  fire  independent  of  any  user  input  (this  makes  sense  due  to 
the  correlation  between  fact-assertion  and  rule- firing).  However,  functionality  of  rules  firing  has 
not  changed  between  Phase  2  and  Phase  3. 

Generate  RDF 

When  the  system  has  determined  that  an  event  should  be  marked  ‘Abnormal’  (for  the  purposes 
of  Phase  3  System,  all  events  are  considered  ‘Abnormal’),  the  event  is  mapped  into  the  RDF  re¬ 
source  ‘ActualEvent’.  This  format  allows  other  performers  to  subscribe  to  consistent  information 
statements.  The  following  figure  represents  an  ‘ActualEvent’  RDF  statement  when  output  to  a 
local  file: 


<?xml  version="l . 0"?> 

<rdf : RDF 

xmlns : protege="http : //protege . Stanford . edu/plugins /owl /protege# " 
xmlns  :  xsp="http :  /  /www.  owl  -ontologies  .  com/2005 /0  8  /  07  /xsp .  owl#" 
xmlns="http : //www. owl -ontologies . com/ Onto logy 12 4  9996752 . owl#" 
xmlns : Percept ion=" http : //ltsn . onr . navy . mil /Percept ion# " 
xmlns : owl2xml="http : //www. w3 . org/2006/12/owl2-xml# " 
xmlns : Onto logy2=" http : //ltsn . onr . navy.mil/Ontology/ " 
xmlns : swrlb="http : //www. w3 . org/2003/ll/swrlb# " 
xmlns : rdf="http : //www. w3 . org/1 999/ 02 /22-rdf -syntax-ns# " 
xmlns : Ontology="http : //ltsn . onr . navy.mil/Ontology . owl# " 
xmlns : owl="http : //www. w3 . org/2002/07 / owl# " 
xmlns : xsd="http : //www . w3 . org/2001/XMLSchema# " 
xmlns : swrl="http : //www. w3 . org/2003/ll/swrl# " 
xmlns : rdf s=" http : //www. w3 . org/2000/01/ rdf- schema# " 
xml : base="http : //www. owl -ontologies . com/ Onto logy 12 4  9996752 . owl"> 
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<owl : Ontology  rdf : about=""> 

<owl : imports  rdf : resource="http : //ltsn . onr . navy . mil /Ontology . owl" /> 

</owl : Ontology> 

<Ontology : ActualEvent  rdf : ID="Metallic_Anomaly"> 

<Ontology2 : latitude  rdf : datatype="http : //www. w3 . org/2001/XMLSchema#f loat " 
>45 . 1</Ontology2 : latitude> 

<Per cept ion : probability 

rdf : datatype="http : / /www. w3 . org/2001/XMLSchema#f loat " 

>0 . 7 5</ Perception : probability> 

<Ontology2 : longitude 

rdf : datatype="http : //www. w3 . org/2001/XMLSchema#f loat " 

>45 . 2</Ontology2 : longitude> 

<Ontology : dateCreated 

rdf : datatype="http : //www . w3 . org/2001/XMLSchema#string" 

>2009/08/11  09:19: 14</Ontology : dateCreated> 

<Ontology : creator  rdf : datatype="http : //www . w3 . org/2001/XMLSchema#string" 
>TEXTR0N1< /Ontology : creator > 

COntology : hasThreatLevel 

rdf : datatype="http : / /www. w3 . org/2001/XMLSchema#int " 

>1< /Ontology : hasThreatLevel> 

<Perception : generationTime 

rdf : datatype="http : / /www. w3 . org/2001/XMLSchema#string" 

>2009/08/11  09:19: 14</Perception : generationTime > 

< /Ontology : ActualEvent > 

</rdf : RDF> 

As  seen  above,  the  RDF  properties  correspond  to  information  related  to  the  event. 

Publish  RDF  to  Sesame  Server 

Rather  than  updating  local  files,  the  most  logical  approach  is  to  directly  publish  an  ‘ActualEvent’ 
statement  to  a  Sesame  server  that  other  performers  can  consume.  RDF  statements  are  only  gen¬ 
erated  and  published  when  an  ‘Abnormal’  event  occurs  in  the  Phase  3  System,  and  as  a  result  of 
this,  the  user  has  no  transparency  beyond  the  System  Console  of  when  RDF  is  being  generated 
and  published  (same  as  the  assertion  of  facts  and  firing  of  rules). 
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Figure  3-9  Published  RDF  Resource  listed  through  Sesame  API 


Significant  effort  has  gone  into  the  development  of  ontologies  to  support  this  architecture  under 
the  direction  of  NRL.  Additionally  ONR  has  established  a  collection  of  potential  data  available, 
as  identified  in  the  Generic  Intelligence  Requirement  Handbook  (GIRH)  and  is  mapping  it  to  the 
various  performer’s  services.  While  The  Textron  team’s  services  are  not  directly  outputting  any 
GIRH  data  directly  for  the  DAD  demonstration,  there  is  a  significant  amount  of  GIRH  data  that 
is  present  in  the  Geo-cultural  Instance  Data  that  could  be  leveraged  in  the  future. 


3.5.3  ISR-to-C2  Demonstration 

Baseline  development  efforts  have  focused  on  the  objective  ONR  architecture  and  participation 
in  the  DAD  demo.  Textron  has  also  been  asked  to  participate  in  the  ISR-to-C2  demonstration  at 
Ft  Smith  in  Oahu  Hawaii  in  September  2009.  The  paradigm  for  interoperability  is  different  than 
in  the  DAD  demonstration  and  requires  using  the  publish  subscribe  architecture  of  the  Raytheon 
Distributed  Knowledge  and  Knowledge  Needs  (DKKN)  system.  As  of  the  date  of  this  report  DKKN 
integration  is  progressing.  The  intent  in  this  demonstration  is  for  geo-cultural  queries  to  be  exe¬ 
cuted  based  on  soldier  I2W  voice-to-text  data  or  direct  queries  or  event  statements  from  the 
Company  Level  Operations  Center  (CLOC)  Figure  3.10  illustrates  the  functionality  of  this  sys¬ 
tem 
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CLOC 

Event/Query 

I2W 

Even  I/Query 

Tttlrtjfl  Krsith  S*i 


Figure  3-10  Phase  3  Usage 


System  Inputs 

The  input  components,  shown  in  figure  3-10,  of  the  ISR-C2  System  are  responsible  for  driving 
the  dynamic  element  of  the  system.  The  interest  events  serve  to  simulate  real-time  events  as  they 
occur.  The  instance  and  geo-databases  serve  to  provide  context  for  the  location  and  associated 
activities  with  the  given  location.  The  rules  file  serves  to  provide  inference  capabilities  for  given 
events.  The  combination  of  these  input  events,  whether  static  or  dynamic,  allows  for  a  responsive 
real-time  system  that  will  provide  inferencing  on  a  given  event  and  return  an  appropriate  output. 

DKKN:  CLOC  Interest  Event  and  Queries 

This  event  will  occur  when  the  CLOC  publishes  an  event  of  interest  or  generates  a  query. 

DKKN:  12  W Interest  Event  and  Queries 

This  event  will  occur  using  the  I2W  voice-to-text  interface  Lockheed  Martin  has  developed. 
There  are  a  number  of  common  phrases  that  a  user  may  speak,  such  as  “Market  is  not  busy,”  that 
will  be  digitized,  recorded,  and  published  to  the  DKKN  server  as  an  interest  event.  Other  queries 
from  the  I2W  device  will  be  supported  as  needed. 

Instance  Database 

Considerable  unclassified  instance  data  has  been  collected  for  the  Garmsir  area  in  Afghanistan. 
This  data  will  be  enhanced  and/or  modified  to  support  the  efforts  of  this  demonstration. 
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Geo-database 

The  specific  content  of  the  geo-database  for  this  demonstration  is  still  being  discussed.  This  will 
change  based  on  the  final  content  of  the  demonstration. 

Rules  File 

The  Rules  file  contains  ‘rules’  that  contain  a  subject  and  a  predicate.  The  subject  is  the  input  to 
the  system,  such  as  a  market  observed  as  being  ‘not  busy’,  and  via  the  associations  drawn  from 
the  Instance  Database  and  Geo-database,  the  predicate,  or  output  of  the  event,  is  that  the  Target 
Level  for  the  given  location  is  increased.  For  the  purposes  of  the  ISR-C2  demonstration,  the  rules 
file  will  contain  only  a  subset  of  the  developed  rules  with  the  intent  of  demonstrating  in¬ 
put/output  functionality  in  conjunction  with  other  performers,  rather  than  advanced  inference¬ 
making. 

Query  Response 

The  ISR-C2  System  Response  details  how  the  system  will  respond  to  any  input.  The  process  in¬ 
volves  assembling  and  asserting  the  pertinent  information  pertaining  to  the  ‘fact’,  leveraging  the 
limited  rules  set  to  fire  a  small  number  of  rules,  and  providing  the  updated  result  set  back  to 
DKKN  for  other  performers  to  subscribe  to. 

Fact  is  Assembled 

When  an  event  of  interest  occurs  and  is  published  to  the  DKKN  server,  the  system  must  respond 
to  it  in  the  same  manner,  no  matter  what  the  source  of  the  event  is.  Thus,  assembling  the  fact  into 
a  meaningful  data  format  is  necessary  to  allow  proper  inferencing. 

Fact  is  Asserted 

Once  the  ‘fact’  has  been  assembled  in  a  manner  that  will  allow  the  inference  engine  to  interpret 
the  information  contained  within,  the  ‘fact’  is  asserted.  The  assertion  process  is  simply  adding 
the  ‘fact’  to  the  initialized  local  Knowledge  Store  (which  also  contains  stateful  intelligence  re¬ 
garding  the  Rules  File). 

Rules  are  Fired 

For  a  given  fact,  a  subset  of  the  total  number  of  rules  will  fire,  where  that  subset  is  between  0  and 
the  total  number  of  rules.  Thus,  though  unlikely,  an  asserted  fact  could  cause  every  single  rule  to 
fire,  or  could  cause  none.  When  a  fact  is  asserted  that  meets  the  criteria  of  the  subject  of  a  rule, 
then  the  predicate  of  the  rule  is  executed.  For  instance,  if  an  ‘Arrest’  event  occurs  (subject),  then 
the  Threat  Level  of  that  person’s  organization,  based  on  their  Socio-Economic  Status  and  previ¬ 
ous  attack  history  will  be  elevated  (predicate).  The  output  of  a  subset  of  rules  firing  is  an  increas¬ 
ing  and  decreasing  of  appropriate  Target  and  Threat  Levels,  which  will  be  cataloged  in  the  ‘Tex¬ 
tron  Result  Set’  and  published  back  to  the  DKKN  server.  A  direct  query  into  the  system  should 
in  theory  fire  0  rules. 

DKKN:  Textron  Result  Set 
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The  Result  Set  is  simply  the  observed  changes  in  the  system  given  the  most  recent  rule  firing. 
This  Result  Set  exists  to  show  the  cause-effect  relationship  between  an  event  occurring  (cause), 
and  the  appropriate  Target  and  Threat  Levels  changing  (effect). 

3.5.4  Anthropology 

Development  has  been  ongoing  in  Phase  3  of  an  expanded  Ontology  that  covers  queriable  An¬ 
thropological  constructs  and  data  types.  A  team  of  subject  matter  experts  have  been  used  to  de¬ 
fine  the  new  Ontology  elements.  They  include: 

•  TUS  team  members  including  a  geographer 

•  An  anthropologist 

•  Former  Army  Infantry  Officer 

•  Former  Army  Intelligence  Officer 

•  Former  Marine  Corp  Intelligence  Officer 

•  Former  Navy  Seal 

•  The  Ontology  has  currently  been  instantiated  in  Protege  and  development  of  linkages  be¬ 
tween  elements  is  ongoing,  to  be  further  defined  in  the  next  phase  of  the  program.  Fig¬ 
ures  3.11  -3.13  illustrate  the  combined  ontology: 
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Figure  3-11  Geo-Cultural  Elements 
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Figure  3-12  Anthropological  Elements 
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Figure  3-13  Expanded  Symbols 
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3.6  Performer  Interaction 

The  Textron  team  has  worked  hard  to  interact  with  other  performers  to  support  information  sharing,  archi¬ 
tecture  interoperability  and  development  and  execution  of  ONR  demonstrations.  These  performers  in¬ 
cluded: 


•  Aptima:  Interoperability,  service  interaction  (event  modeling)  and  DAD  demo 

•  SAIC:  Sensors 

•  Toyon:  Sensors 

•  Eureka:  Sensors 

•  Chi:  Geo-cultural 

•  Raytheon:  DKKN  and  ISR-C2  demo 

•  LM-ATL:  I2W 

•  BBN:  Loan  of  UGS  platforms  and  ISR-C2  demo 

•  SPADAC:  Sensor  placement  and  Cultural 

•  CCRI:  Joint  visit  to  NRL  for  DAD  and  interoperability 

•  Metron:  Joint  visit  to  NRL  for  DAD  and  interoperability 

Additional  Textron’s  team  has  been  actively  participating  in  the  TWiki  on  a  daily  basis  for  the 
last  8  months.  They  have  participated  in  the  following  WebTopics 

•  PerceptionOntologyTutorial 

o  Actively  participated  in  the  perception  ontology  web  topic  which  was  the  first  on¬ 
tology  that  members  collaborated  on 

•  InformationVoidOntology  (later  SensorTaskingOntology) 

o  Developed  and  submitted  a  sensor  tasking  ontology  which  is  included  in  the  latest 
version  of  the  ontology 

•  EventUseCase 

o  Developed  an  event  use  case  for  others  to  leverage  when  generating  RDF. 
o  Researched  generating  RDF  from  the  Protege  API’s  programmatically  instead  of 
using  string  builders  to  produce  XML.  This  gives  us  scalability  when  generating 
RDF  in  the  future 

•  DiscussionOfRequiredEventProperties 

o  Actively  discussed  and  collaborated  with  Aptima  and  Metron  regarding  the  gen¬ 
eration  of  events.  Ultimately  a  simplistic  approach  was  taken  to  the  eventing  as 
the  data  is  still  a  mystery  to  most, 

Textron  has  been  very  active  in  the  TWiki  process  and  has  been  vocal  on  many  topics  related 
and  unrelated  to  Textron’s  RDF  output. 
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3.7  Testing 

Testing  activities  are  documented  in  the  Experimentation  Test  Plan,  an  update  of  which  is  being  delivered 
prior  to  this  document  (see  section  2.3). 

3.8  Demonstrations 

Demonstrations  were  provided  for  TIM1  and  TIM2  and  are  documented  in  the  meeting  minutes  for  each 
TIM  (see  section  2.3).  Ongoing  efforts  to  support  the  Data  Analysis  Demo  DAD  and  the  ISR-to-C2 
Demo  will  be  concluded  in  the  next  phase  of  the  program  and  are  discussed  in  section  3.5.  2  and  3.5.3 
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4  Results,  Conclusions  and  Next  Steps 

The  base  year  of  the  Textron  Transparent  Urban  Structures  effort  was  an  evolutionary  path  of  developing 
increasing  functionality  in  support  of  a  very  dynamic,  very  interactive  architecture  and  surrounding  com¬ 
munity.  After  this  year  the  overall  goals  of  the  combined  performers  have  become  much  clearer.  Addi¬ 
tionally  the  individual  interactions  between  specific  performers  are  becoming  apparent.  Jointly  developed 
and  defined  ontologies  are  allowing  common  views  of  large  amounts  of  contextual  data.  Finally  the  stan¬ 
dards  required  to  implement  the  architecture  are  beginning  integration  into  each  performers  environment, 
enabling  what  is  about  to  become  the  DAD  demonstration  and  the  C2-ISR  demonstration. 

For  Textron  each  phase  had  a  different  set  of  goals.  Phase  1  provided  refinement  of  the  architecture  and 
implement  of  the  framework  and  infrastructure  need  to  support  the  next  phases.  Phase  2  provided  the 
integration  of  the  geo-cultural  service,  the  first  attempt  at  an  RBIE,  the  development  of  rules  and  the  de¬ 
velopment  of  an  experimentation  environment  and  scenario.  Phase  3  involved  developing  interfaces  to 
enable  integration  with  other  performers  and  the  ONR  architecture,  development  of  performer  wide  on¬ 
tologies,  development  of  an  initial  anthropological  ontology  and  collaboration  in  support  of  the  DAD  and 
ISR-to-C2  demonstrations. 

As  the  option  year  begins,  a  series  of  goals  are  in  front  of  this  team  in  order  to  develop  a  better  more  in¬ 
teroperable  system.  They  include: 

■  Successful  demonstration  with  DAD  and  ISR-to-C2  efforts 

■  Advanced  anthropology  ontology  and  query  capabilities 

■  Further  development  in  interfaces  and  standards 

■  Broader  inferencing  capabilities  for  multiple  environments 

■  Larger  testbed  capabilities  using  existing  Iraq  and  Afghanistan  data  sets 

■  Sensor  planning  capabilities 

■  Data  confidence  assessment 

■  Stronger  ties  to  and  collaboration  with  other  performers 
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5  Directory 

5.1  Glossary 

Table  5-1:  Glossary 


Term 

Definition 

ActiveMQ 

The  most  popular  COTS  middleware  software  which  enables  communi¬ 
cation  of  loosely  coupled  systems  via  the  JMS  standard. 

Drools 

Rules  based  inferencing  and  event  processing  engine 

Jess 

JAVA  based  inferencing  engine 

MAK 

Is  a  suite  of  products  which  include  visualization,  simulation  and  link 
analysis  tools.  We  will  be  using  the  simulation  tools  provided  by  back  to 
integrate  real  hardware  into  the  system. 

Ontology 

A  formal  representation  of  a  set  of  concepts  within  a  domain  and  the 
relationships  between  those  concepts 

Protege 

A  suite  of  tools  to  construct  domain  models  and  knowledge-based  appli¬ 
cations  with  ontologies,  developed  by  Stanford  University 

Rete 

Speed  optimizing  forward  chaining  rule  algorithm 

Sesame 

Open  source  Java  framework  for  storing,  querying  and  reasoning  with 
RDF  and  RDF  Schema 

Service  Oriented 

Architecture 

A  service-oriented  architecture  is  a  collection  of  software  services.  These 
services  communicate  with  each  other  either  by  simple  passing  data  or  it 
could  involve  two  or  more  services  coordinating  some  activity.  Some 
means  of  connecting  services  to  each  other  is  needed. 

TWiki 

A  structured  wiki,  typically  used  to  run  a  collaboration  platform, 
knowledge  or  document  management  system,  a  knowledge  base,  or  team 
portal 

Warfighter 

The  Warfighter  is  a  term  used  to  identify  coalition  forces  in  this  urban 
environment. 

Web  Services 

A  software  system  designed  to  support  interoperable  machine-to-machine 
interaction  over  a  network.  It  has  an  interface  described  in  a  machine- 
processable  format. 
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5.2  Acronym  List 


Table  5-2 

:  Acronym  List 

API 

Application  Programming  Interface;  Application  Program  Interface;  Application  Programmer 
Interface 

C2 

Command  and  Control 

CAP 

Common  Application  Framework 

CLOC 

Company  Level  Operations  Center 

DAD 

Data  Analysis  Demo 

DKKN 

Distributed  Knowledge  and  Knowledge  Needs 

EDA 

Event  Driven  Architecture 

ERDC 

Engineering  Research  and  Development  Center  -  US  Army  Corp  of  Eng. 

ETP 

Experimentation  and  Test  Plan 

GCAT 

Geo  Cultural  Analysis  Tool 

GIRH 

Generic  Intelligence  Requirement  Handbook 

GUI 

Graphical  User  Interface 

HITL 

Hardware-in-the-Loop 

HLA 

High  Level  Architecture 

ISR 

Intelligence  Surveillance  and  Reconnaissance 

JMS 

Java  Messaging  Service 

M&S 

Modeling  and  Simulation 

OWL 

Web  Ontology  Language 

RBIE 

Rules  Based  Inferencing  Engine 

RDF 

Resource  Description  Framework 

SIC 

Sensor  Information  Collection  Service 

SITL 

Software-in-the-Loop 

SME 

Subject  Matter  Experts 

SOA 

Service  Oriented  Architecture 

SOAP 

Simple  Object  Access  Protocol 

SWRL 

Semantic  Web  Rule  Language 

TIM 

Technical  Interchange  Meeting 

TUS 

Transparent  Urban  Structures 

UGS 

Unattended  Ground  Sensor 

XML 

Extensible  Markup  Language 
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